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Description 

FIELD OF THE INVENTION 

The present invention relates to training method ap- 
plied to a data processing system according to the con- 
cept of a neural network. 

BACKGROUND OF THE INVENTION 

The neural network according to a data processing 
system of thistype is structured with layers by preparing 
neuron model (hereinafter called "neuron") 1, shown in 
Fig. 3, in parallel. Neurons in each layer is synapse con- 
nected to all neurons in other adjacent layers, so as to 
input and output data. According to neuron 1, data 0 is 
output in accordance with the comparison result be- 
tween the sum of multiplied input data from outside 11 , 
1 2, I3...ln by weights W1 , W2, W3...Wn and threshold 6. 

Various comparison methods are possible. For ex- 
ample, when normalization function 1[f] is applied, out- 
put data 0 is express as follows: 

0=1[£Wn-ln-e] (1) 

That is; when ZWn In exceed threshold 9, the neuron 
ignites so that output data 0 becomes "1"; and when 
EWn- In is smaller than threshold 9, output data becomes 
"0". 

A conventional neural network is structured to form 
neural layers by preparing above neurons in parallel and 
to connect the above neural layers in series. Neural lay- 
er, for example, comprises 3 layers; input layer, middle 
layer and output layer, as it is described by Perceptrons 
proposed by Rosenblatt, in which neurons of each layer 
are synapse connected to all neurons of other adjacent 
layer. 

SUMMARY OF THE INVENTION 

According to the above data processing system, the 
operation of weight optimization of synapse of each neu- 
ron is called "learning". The improvement of the efficien- 
cy and the guarantee of the realization of above are the 
important subjects to be investigated. For example, as 
to a back propagation method, which came to public no- 
tice recently, the problems of the escapement from local 
minimum and a convergent time are left. The above ten- 
dency is large especially With respect to the neural net- 
work including a plurality of middle layers. 

The present invention is invented so as to solve the 
above problems of the prior art and has an object to pro- 
vide a learning method performable of an efficient learn- 
ing according to the middle layers. 

A learning method according to the present inven- 
tion is characterized as set out in claim 1 . 

It is essential to increase middle layers for many da- 



ta processings. As the calculation methods for the near- 
est approximation, there are the minimum square ap- 
proximation, etc. 

5 BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 shows a rough structural diagram of a char- 
acter recognition system according to an embodiment 
of the present invention. 
io Fig. 2 shows a schematic diagram of neurons in an 
input layer, a middle layer and an output layer. 
■ Fig. 3 shows a schematic diagram of a neuron. 
Fig. 4 is a graph showing the relationship between 
the number of learnings and weight change. 
is 1 and N... neuron, 31 ...input layer, 32. ..middle layer, 
33.. .output layer. 

PREFERRED EMBODIMENT OF THE PRESENT 
INVENTION 

20 

Hereinafter, the present invention is described ac- 
cording to the embodiments with referring to the at- 
. tached drawings. 

Fig. 1 shows a character recognition system com- 

2S prising a data processing system according to a learning 
method of an embodiment of the present invention. This 
character recognition system comprises: a video cam- 
era 10, a preprocessing system 20, a data processing 
system 30, a post-processing system 40 and display 50. 

30 A video camera 1 0 is prepared for inputting characters 
and is connected to a preprocessing system 20. A pre- 
processing system 20 is, for example, an image 
processing system, is generally and conventionally 
known, that extracts the characteristics data of input 

35 character (for example, number of end points, number 
of branch points, etc.) and outputs this characteristics 
data to a data processing system 30. A data processing 
system 30 structures the neural network, as it is de- 
scribed later, and outputs data to a post-processing sys- 

40 tern 40 according to the recognition result by recogniz- 
ing an input character from a preprocessing system 20 
according to its characteristics data. Recognition sign 
is, for example, a character code. A post-processing 
system 40 stores the above output data as, for example, 

45 a data of a work processor, and outputs it to a display 
20, simultaneously. Display 50, for example, of the CRT 
type, displays recognized character by a data process- 
ing system 30 onto an image. 

The neural network structuring a data processing 

so system 30 is structured as a part of hardware. This data 
processing system 30, typically indicated in Fig. 1 , com- 
prised input layer 31 , middle layer 32 and output layer 
33 in which middle layer 32 is arranged between input 
layer 31 and output layer 33, as it is understood from 

55 this figure. According to the present embodiment, each 
layer 31 , 32 and 33 comprises an equivalent number of 
neurons N, in which neurons N of input layer 31 are con- 
nected to all neurons N of middle layer 32, and neurons 
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N of middle neural layer 32 is connected to all neurons 
N of output layer 33. 

Each neuron N outputs either "1" or "0" according 
to the formula (1) of the normalization function, as it is 
described with referring to Fig. 3. Neuron N is imple- 
mented, for example, by operation amplifiers. Weight 
Wn to be multiplied to a data input to each neuron N is 
obtained, for example, by a variable resistance connect- 
ed to an input terminal of an operation amplifier. Thresh- 
old function is realized by, for example, switching ele- 
ments. Accordingly, weight Wn is changed and output 
data is outputted by changing variable resistance with 
respect to the output data of each neuron, so as to per- 
form the learning control. 

Fig. 2 shows typical input layer 31 , middle layer 32 
and output layer 33, comprising equivalent number of 
neurons among themselves. Here, it is determined that 
number of neurons in each layer 31 , 32 and 33 is 36, for 
the simple explanation, in which 6 neurons are arranged 
in horizontal direction and 6 neurons are arranged in 
vertical direction. When the location of a neuron at the 
down left comer in this figure is determined as an origin, 
a neuron at the i-th and j-th location from the left and 
from the bottom, respectively, is determined as Nij. 

Each neuron of an input layer 31 ignites with respect 
to characteristics data of character input through a video 
camera 10. For example, it is deemed that number of 
end points is expressed by ignition pattern of N11 , N12, 
N21 and N22; and numberof branch points is expressed 
by ignition patterns of N 13, N 14, N23 and N24. Accord- 
ingly, ignition pattern of neurons of an input layer 31 is 
determined in accordance with an input character, arti- 
ficially. 

On the other hand, at on output layer 33, an char- 
acter is expressed, for example, by neuron Nii on a di- 
agonal line connecting neurons at down left corner and 
down right comer. Accordingly, neurons on the above 
diagonal line express character codes of a character. 
The ignition pattern of neurons of an output layer 33 is 
determined artificially. According to the present embod- 
iment, the number of ignition patterns of neuron Nii on 
the diagonal line is 64. Therefore, it is possible to rec- 
ognize 64 kinds of characters, according to the present 
embodiment, so that recognition of, for example, the al- 
phabet is possible. 

Before learning of character recognition is per- 
formed, neurons of output layer 33 do not ignite even if 
character data in input to a data processing system 30. 
The ignition of a neuron makes learning possible, and 
learning will be completed when a predetermined igni- 
tion pattern is output in accordance with input character 
data. The input and output patterns for learning is the 
representative data to be processed at the neural net- 
work. Data to be processed actually covers a wide area. 
Learning is continuously executed until a suitable asso- 
ciation is performed with respect to the above represent- 
ative input and output data. Therefore, when learning is 
completed, input layer 31 and output layer 33 output ar- 



tificially determined ignition pattern, as it is described 
above, according to input character data. Then it is as- 
sumed that the ignition patterns of input layer 31 , middle 
layer 32 and output layer 33 are changed smoothly over 

5 the above layers 31 , 32 and 33. According to the present 
embodiment, weights of middle layer 32 and output lay- 
er 33 are changed on a process of learning so that the 
ignition patterns of the above layer change smoothly. 
Here, the location of a neuron at down left corner is 

10 determined as on origin, and an output data of neuron 
Nij of an input layer 31 at the i-th location from left and 
from down, respectively, is determined as aij. Similar to 
the above, output data of neuron Nij of a middle layer 
32 is determined as bij; and output data of neuron Nij of 

is ' an output layer 33 is determined as cij. 

The sum of squares of the differences among each 
neuron of each layer 31 , 32 and 33 is expressed as fol- 
lows: 

f=L{(aij-bij) 2 +(bij-cij) 2 } 

Accordingly, the above sum of squares f is the square 
distance with respect to the relationship between igni- 
2s tion patterns. Here, £ indicates that i and j take the sum 
from 1 to 6. 

Once an input character is determined, aij and cij 
are fixed, but not bij. Therefore, bij, when the sum of 
squares f is the minimal value, is easily calculated by, 
30 for example, a method of the least squares as is gener- 
ally know. According to the learning of character recog- 
nition, weights of each neuron of a middle layer 32 and 
a output layer 33 are increased by predetermined value 
so as to obtain bij, i.e. ignition pattern, obtained as 
.35 above. Hereinafter, the above processing is described 
with referring Fig. 3. Weights of synapses (in this case, 
"W2" and "3") is, for example, increased by 5% with re- 
spect to input data from ignited neuron (for example, "12" 
and "13") among neurons of input layer connected with 
40 neuron 1 when output data O(bij) of neuron 1 of a middle 
layer is the value at the ignition (for example "1"). Ac- 
cording to weights of synapse with respect to the neu- 
rons of output layer, it is processed as similar to the 
above. As it is described above, weights of synapse are, 
45 for example, increased by 5% with respect to neurons 
of a middle layer that is determined to be ignited by a 
method of least squares. 

Accordingly, weights of a middle layer 32 and an 
output layer 33 are increased so that the ignition pat- 
so terns are smoothly changed over layers of an input layer 
31 , a middle layer 32 and an output layer 33. Here, when 
each neuron of an input layer 31 and output layer 33 is 
determined to ignite with closer frequency as possible 
with respect to all input character, the equivalent ignition 
55 among each neuron of a middle layer 32 becomes pos- 
sible. Accordingly, it is possible to prevent falling into a 
local minimum. Furthermore, each neuron of a middle 
layer 31 will ignite approximately equivalently. There- 
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fore, evasion of the occurrence of neurons, which do not 
ignite, is possible so as to function the neurons of a mid- 
dle layer 32 efficiently. 

The weight increase of synapse for a learning 
change as shown in Fig. 4 with respect to the number 
of learnings. The learning for a whole system is gradu- 
ally performed by a plurality of learnings, and a slight 
adjustment is performed with a little change at the end 
of learning. Learning speed is heightened as weight in- 
creases rapidly at the beginning of learning. 

According to the above embodiment, there are 3 
layers; a middle layer 32 are connected to an input layer 

31 and an output layer 33. However, the present inven- 
tion can be applied not only to the neural network with 
3 layers but also the neural network with more then 3 
layers. In this case, one middle layer should be selected 
first, then weights of synapses of neurons at the middle 
layer and an output layer are to be increased so that the 
sum of squares of the ignition patterns becomes the 
minimum value between that of in a middle layer and in 
an input layer 31 and output layer 33. After completing 
the learning with respect to all input characters up to the 
predetermined level, the next learning is performed by 
increasing the weights of synapses of neurons in the 
second middle layer and an output layer so that the sum 
of squares of the ignition pattern of the second middle 
layer becomes the minimum according to the ignition 
patterns of the other 2 layers adjacent to the above mid- 
dle layer (the first middle layer and an input layer or an 
output layer). Accordingly, weights distribution with re- 
spect to the neural network with 4 layers is obtained. 
The same processing manner is applied to the neural 
network with more than 4 layers. 

The method for deciding weights distribution of mid- 
dle layer 32 and output laye r 33 can be conceivable var- 
ious method other than the first embodiment above. 

For example, the absolute value of the difference of 
output data of neuron of each layer can be considered. 
That is, "f" of Hamming distance is considered, which is 
the accumulated value of laij-bijl and Ibij-cijl regarding 
all neurons. r . 

f=S{laij-bijl+lbij+cijl} 

Here, laij-bijl is the absolute value of the difference be- 
tween an output data of a neuron of input layer 32 and 
an output data of a neuron of middle layer 32, and 
Ibij+cijl is the absolute value of the difference between 
an output data of middle layer 32 and an output data of 
output layer 33. In this embodiment, " " is the accumu- 
lation from 1 to 6 with respect to T and "j". 

"aij" and "cij" are fixed when input character is de- 
cided, but only "bij" is not fixed. Therefore, "bij" on the 
minimum value of Hamming distance "f" can be easily 
found out by general method. On learning of character 
recognition, the weights of each neuron of middle layer 

32 and output layer 33 are increased by predetermined 



value in order to obtain "bij" of ignition pattern. 

This is explained below referring Fig. 3. When out- 
put data "O(bij)" of the middle layer is the ignition value 
("1", for example), synapse weight ("W2" and "3") cor- 

5 responding to input data from ignited neuron ("12" and 
"13", for example) among the neurons of input layer con- 
nected to neuron 1 is increased 5%, for example. The 
weight of synapse of a neuron of output layer is proc- 
essed in the same way, therefore, the weight of synapse 

10 corresponding to the neuron of the middle layerdecided 
to be ignited from Hamming distance is increased 5%, 
for example. 

The same effect of the first embodiment can be ob- 
tained by the second embodiment, too. 

is The second embodiment can be also applied to a 
neural network with neural layers of 4 or more, other 
than the neural network with 3 layers, similarly to the 
first embodiment. In this case, selecting the first middle 
layer, the weight of the synapses in the first middle layer 

20 and output layer are increased in order for the sum to 
be the minimum value, that is, in order for Hamming dis- 
tance to be the minimum value: here, the "sum means 
the total value of the absolute value of the difference 
between the output data of each neuron of input layer 

25 31 and that of the first middle layer, and the absolute 
value of the difference between the output data of each 
neuron of the first middle layer and that of output layer 
33. When the learning up to the predetermined stage is 
completed with respect to all input characters, the sec- 

30 ond middle layer is newly added and the weight of the 
synapse is increased in the second middle layer and the 
layer connected to the output side of the second middle 
layer (output layer or the first middle layer), in the same 
way of the first middle layer in order for Hamming dis- 

35 tance to be the minimum value. Consequently, weight 
distribution for 4 layers can be obtained. The same 
processing is executed for the case with 5b layers or 
more. 

The third embodiment of the present invention is de- 
40 scribed next. 

The coordination of neuron Nij in each layer 31 , 32 
and 33 is the same and they are corresponding to one 
another. 

In order for ignition pattern of each layer of 31 , 32 
45 and 33 to change smoothly, the neuron of middle layer 
32 has the tendency to ignite when corresponding neu- 
ron in input layer 31 and output layer 33 both ignite. 
Therefore, in this embodiment, the weight of synapse of 
predetermined neuron of middle layer 32 and output lay- 
so er 33 is increased by the predetermined rate in order for 
each neuron in middle layer 32 ignite easily when the 
neuron of input layer 31 and corresponding neuron in 
output layer 33 are both ignite. 

This is explained referring to Fig. 2 and Fig. 3. In 
55 this explanation, neuron Nij of output layer 33 is as- 
sumed to be the construct of a part of character code. 

When neuron Nij in input layer 31 and neuron Nij in 
output layer 33 are both to be ignited for recognizing a 
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character, the weight distribution of synapses of middle 
layer 32 and output layer 33 so that neuron Nij in middle 
layer 32 ignite easily. For example, when neuron 1 in 
Fig. 3 is a neuron Nij of middle layer 32, the weight of 
the synapse corresponding to the input data from ignited 
neuron among the neurons of input layer connected to 
neuron 1 is increased by 5%. When input data "12" and 
"13" ignite, the weight of the synapse "W2" and "W3" are 
increased. The same processing is performed to the 
weight of synapse of neuron of output layer, that is, the 
weight of the synapse corresponding to the neuron to 
be ignited in middle layer is increased by 5%, for exam- 
ple. 

From the third embodiment, the same effect of the 
first and the second embodiments can be obtained, too. 

The third embodiment can be also applied t6o the 
neu ral network with 4 layers or more, other than the neu- 
ral network with 3 layers, as the first and the second 
embodiments. In this case, selecting the first middle lay- 
er, the weight of the synapse of the middle layer and the 
output layer is increased so that each neuron in the mid- 
dle layer obtains the tendency to ignite when the corre- 
sponding neurons in input layer and output layer are 
both ignite. When the learning up to the predetermined 
stage is completed, the second middle layer is added 
and the weight of synapse is increased in the second 
middle layer and the layer connected to the output side 
of the middle layer (output layer or the first middle layer) 
in the same way as the first middle layer. In this way, the 
weight distribution if obtained in the case of 4 layers. 
The same processing is executed in the case of more 
than 5 layers. 

The fourth embodiment of the present invention is 
explained below. 

It is considered that the ignition patterns of the lay- 
ers 31 , 32 and 33 change smoothly when the neuron in 
middle layer 32 has the tendency to ignite only in the 
case that at least one of corresponding neurons in input 
layer 31 and output layer 33 ignites. Therefore, in this 
embodiment, the weight of synapse of predetermined 
neuron is increased by predetermined rate in order that 
each neuron in middle layer 32 ignites easily when at 
least one of corresponding neurons in input layer 31 and 
output layer 33. 

This is described in detail referring Fig. 2 and Fig. 
3. A neuron Nij in output layer 33 is assumed to be a 
constituent part of a character code, as the case of the 
third embodiment. 

When at least one of neurons Nij in input layer 31 
and output layer 33 is to ignite for recognizing a charac- 
ter, the weight distribution of synapses in middle layer 
32 and output layer 33 is to be decided in order for neu- 
ron Nij in middle layer 32 ignites easily. For example, 
when neuron 1 in Fig: 3 is the neuron Nij of middle layer 
32, the weight of the synapse of input data from ignited 
neuron among the neurons in input layer connected to 
neuron 1 is increased by 5%. When input data is "12" 
and "13", the weight of the synapse to be increased 



weight is "W2" and "W3". The weight of the synapse of 
the neuron in output layer is performed the same 
processing. As described above, the weight of synapse 
of the neuron in middle layer to ignite is increased by 

s 5%, for example. 

From the fourth embodiment, the same effect of the 
embodiments above can be obtained. 

The fourth embodiment can be also applied to the 
neural network with 4 layers or more, other than the neu- 

io ral network with 3 layers. In this case, selecting the first 
middle layer, the weight of the synapse of the middle 
layer and output layer is increased so that each neuron 
in the middle layer obtains the tendency to ignite when 
at least one neuron of the corresponding neurons in in- 

15 put layer and output layer ignites. When the learning up . 
to the predetermined stage is completed, the second 
middle layer is added and the weight of synapse is in- 
creased in the second middle layer and the layer con- 
nected to the output side of the middle layer (output layer 

20 or the first middle layer) in the same way as the first mid- 
dle layer. In this way, the weight distribution is obtained 
in the case of 4 layers. The same processing is executed 
in the case of more than 5 layers. 

Input/output data for a neuron does not have to be 

25 the binary data. Multi-valued data and analog data are 
also acceptable. 

Furthermore, an output layer is not necessary struc- 
tured by the neurons on a diagonal line to express the 
character codes; it is possible to be structured by all neu- 

30 rons to define the recognition characters. 

Number of neurons for each layer 31 , 32 and 33 is 
not limited to 36 as the present embodiment. As many 
necessary neurons as the number of kinds of character 
to be recognized can be prepared. 

35 it is possible to apply the present invention not only 
to the character recognition but also to the configuration 
recognition or to acoustic recognition. 

According to the present invention as mentioned 
above, it is possible to obtain an effect that an efficient 

40 learning can be performed onto the middle layer of the 
neural network. 



Claims 

1. A training method for transforming an untrained net- 
work into a trained network, said network compris- 
ing an input layer (31), a middle layer (32) and an 
output layer (33), each said layer including a plural- 

so ity of neurons, each said neuron being a signal 
processing element, said middle layer being ar- 
ranged between said input and output layers said 
method being characterised in that each neuron in 
said middle layer is connected to each neuron in 

55 said input and output layers, and by the steps of: 

determining an activation pattern of said neu- 
rons in said input layer; 
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determining an activation pattern of said neu- 
rons in said output layer; 

for each neuron of the middle layer increasing 
the weight by which connections between neu- 
rons of the input layer and said neurons in the 
middle layer and increasing the weights corre- 
sponding to connections between said neuron 
of the middle layer and neurons of the output 
layer if this increasing contributes to a better 
approximation of the ignition pattern of the mid- 
dle layer to the ignition patterns of the input lay- 
er and the output layer; and 

repeating the increasing step for each neuron 
of the middle layer, thereby generating the 
nearest 

approximation of the ignition pattern of the mid- 
dle layer to the ignition patterns of the input lay- 
er and the output layer. 

A training method according to claim 1, further com- 
prising the steps of: 

adding an additional middle layer between said in- 
put layerand said middle layeror between said mid- 
dle layer and said output layer and performing the 
increasing step for the additional middle layer in the 
same manner as for the first middle layer. 

A training method according to claim 1 or 2, wherein 
the increasing of the weights is performed in order 
to minimize the sums of corresponding absolute dif- 
ferences between the output values of neurons of 
the input layer and the middle layer and between 
the output values of the middle layer and the output 
layer. 

A training method according to claim 1 or 2, wherein 
the increasing of the weights for a certain neuron of 
the middle layer is performed when the correspond- 
ing neurons in the input and output layers both ig- 



A training method according to claims 1 or 2, where- 
in the increasing of the weights for a particular neu- 
ron of the middle layer is performed when either of 
the corresponding neurons of the input layer and 
the output layer ignites. 



Patentanspruche 

1. Trainingsverfahren zum Umwandeln eines untrai- 
nierten Netzwerkes in ein trainiertes Netzwerk, wo- 55 
bei das Netzwerk eine Eingangsschicht (31), eine 
Mittelschicht (32) und eine Ausgangsschicht (33) 



ronen aufweist und jedes Neuron ein Signalverar- 
beitungselement darstellt, wobei die mittlere 
Schicht zwischen der Eingangs- und Ausgangs- 
schicht angeordnet ist, und wobei das Verfahren 
dadurch gekennzeichnet ist, da(3 jedes Neuron in 
der mittleren Schicht mit jedem Neuron in der Ein- 
gangs- und Ausgangsschicht verbunden ist sowie 
durch folgende Schritte: 

Bestimmen eines Zundungsmusters der Neu- 
ronen in der Eingangsschicht; 

Bestimmen eines Zundungsmusters der Neu- 
ronen in der Ausgangs schicht; 

fur jedes Neuron der mittleren Schicht Erhohen 
der Gewichte der Verbindungen zwischen Neu- 
ronen der Eingangsschicht und der Neuronen 
in der mittleren Schicht und Erhohen der Ge- 
wichte der Verbindungen zwischen dem Neu- 
ron der mittleren Schicht und den Neuronen der 
Ausgangsschicht, falls die Erhohung zu einer 
besseren Annaherung des Zundmusters der 
mittleren Schicht zu den Zundmustern der Ein- 
gangsschicht und der Ausgangsschicht fuhrt, 
und 

Wiederholen des Erhohungsschrittes fur jedes 
Neuron der mittleren Schicht, wodurch die be- 
ste Aproximation des Zundungsmusters der 
mittleren Schicht zu den Zundungsmustern der 
Eingangsschicht und der Ausgangsschicht er- 
zeugt wird. 

Trainingsverfahren nach Anspruch 1 , weiterhin auf- 
weisend die Schritte: 

Addieren einer zusatzlichen mittleren Schicht zwi- 
schen der Eingangsschicht und der mittleren 
Schicht oder zwischen der mittleren Schicht und der 
Ausgangsschicht und Ausfuhren des Erhohungs- 
schrittes fur die zusatzliche mittlere Schicht in der 
gleichen Weise wie fur die erste mittlere Schicht. 

. Trainingsverfahren nach Anspruch 1 oder 2, wobei 
das Erhohen der Gewichte ausgef uhrt wird, urn die 
Summe der entsprechenden absoluten Differenzen 
zwischen den Ausgangswerten der Neuronen der 
Eingangsschicht und der mittleren Schicht und zwi- 
schen den Ausgangswerten der mittleren Schicht 
und der Ausgangsschicht zu niinimieren. 

. Traningsverfahren nach Anspruch 1 oder 2, wobei 
das Erhohen der Gewichte fur ein bestimmtes Neu- 
ron der mittleren Schicht ausgefuhrt wird, wenn die 
entsprechenden Neuronen in der Eingangs- und 
Ausgangsschicht beide zunden. 

aufweist, wobei jede Schicht eine Vielzahl von Neu- 5. Trainingsverfahren nach Anspruch 1 oder 2, wobei 



6 



EP 0 450 521 B1 



12 



das Erhohen der Gewichte fur ein bestimmtes Neu- 
ron der mittlerenSchicht ausgefuhrt wird, wenn ent- 
weder die entsprechenden Neuronen der Ein- 
gangsschicht oder der Ausgangsschicht zunden. 



Revendications 

1. Procede d'apprentissage pour transformer un re- 
seau non forme en un reseau forme, ledit reseau 
comprenant une couche d'entree (31), une couche 
mediane (32) et une couche de sortie (33), chacune 
desdites couches contenant une pluralite de neuro- 
nes, chaque neurone etant un Element de traits- 
ment de signaux, ladite couche mediane etant pla- 
ced entre lesdites couches d'entree et de sortie, le- 
dit procede etant caracterise en ce que chaque neu- 
rone de ladite couche mediane est connecte a cha- 
que neurone desdites couches d'entree et de sortie 
et par les etapes consistant a : 



tie de la couche mediane et de la couche de sortie. 

4. Procede d'apprentissage selon la revendication 1 
ou 2, dans lequel I'augmentation des poids pour un 

5 certain neurone de la couche mediane est effectuee 
quand les neurones correspondants de la couche 
d'entree et de la couche de sortie sont tous deux 
excites. 

5. Procede d'apprentissage selon la revendication 1 
ou 2, dans lequel I'augmentation des poids pour un 
neurone particulier de la couche mediane est effec- 
tuee quand I'un ou I'autre des neurones correspon- 
dants de la couche d'entree et de la couche de sor- 
tie est excite. 



determiner une configuration de I'activation 
desdits neurones de ladite couche d'entree ; 
determiner une configuration de I'activation 
desdits neurones de ladite couche de sortie ; 
pour chaque neurone de la couche mediane, 
augmenter les poids des connexions entre des 
neurones de la couche d'entree et lesdits neu- 
rones de la couche mediane et augmenter les 
poids correspondants des connexions entre le- 30 
dit neurone de la couche mediane et des neu- 
rones de la couche de sortie si cette augmen- . 
tation contribue a une meilleure approximation 
entre le diagramme d'excitation de la couche 
mediane et les configurations d'excitation de la 3S 
couche d'entree et de la couche de sortie ; et 
repeter I'etape d'augmenta'tion pour chaque 
neurone de la couche mediane en creant ainsi 
('approximation la plus proche de la configura- 
tion d'excitation de la couche mediane par rap- 40 
port aux configurations d'excitation de la cou- 
che d'entree et de la couche de sortie. 



2. Procede d'apprentissage selon la revendication 1 , 
comprenant encore les etapes consistant a : 45 

ajouter une couche supplemental mediane 
entre ladite couche d'entree et ladite couche media- 
ne ou entre ladite couche mediane et ladite couche 
de sortie et effectuer I'etape d'augmentation pour la 
couche mediane supplemental de la meme ma- so 
niere que pour la premiere couche mediane. 

3. Procede d'apprentissage selon la revendication 1 
ou 2, dans lequel I'augmentation des poids est ef- 

: fectuee dans le but de minimiser les sommes des ss 
differences absolues correspondantes entre les va- 
leurs de sortie des neurones de la couche d'entree 
et de la couche mediane et entre les valeurs de sor- 
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